EP 0 713 338 A2 


(id 


EP 0 713 338 A2 


/ 




EuropSIsches Ra t e n t amt 
European Patent Office 


Office europden des brevets 


(1 2) EUROPEAN PATENT APPLICATION 

(43) Date of publication: (51) bit Cl. 6 : H04N 7/24 

22.05.1996 Bulletin 1996/21 

(21) Application number: 95306065^4 

(22) Date of filing: 30.08.1995 


(84) Designated Contracting States: 

• Ctvanlar, Mehmet Reha 

DE GB 

Red Bank, New Jersey 07701 (US) 

(30) Priority: 18.11.1994 US 341787 

(74) Representative: Buckley, Christopher Simon 
Thlrsk et al 

(71) Applicant: AT&T Corp. 

AT&T (UK) LTD., 

New York, NY 10013-2412 (US) 

AT&T Intellectual Property Division, 
5 Momlngton Road 

(72) Inventors: 

• Cash, Glenn Lawrence 
Matawan, New Jersey 07747 (US) 

Woodford Green, Essex IG8 0TU (GB) 


(54) Method of and apparatus tar video bitstream transmission over packet networks 


(57) Apparatus for and method of sending a video 
bitstream, inducting a plurality of high priority segments 
and assodated low priority segments, by transmitting 
only the low priority segments from a transmitter over a 
faddy to a receiver is disclosed. The transmitter and 
receiver locations are arranged to use (130) previously 
agreed-to high priority information (eg., predefined high 
priority segments or format for generating same) and, 
consequently, only the low priority segments of the video 
bitstream need to be transmitted (134) to the receiver. At 
the receiver, the high priority segments are obtained 
(from storage or generated using the agreed-to format) 
and interleaved in real time with the received low priority 
segments to recreate an interleaved video bitstream. 
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Description 
Technical Held 

This invention relates to video transmission aid, 
more particularly, to a technique for transmitting com- 
pressed previously stored video ova lossy packet net- 
works in real time so as to reduce the effects of packet 
losses on the quality of the received video 

Background of the Invention 

Transmission of stored compressed video over lossy 
packet networks has a number of important applications 
including movies-on-demand, interactive television, and 
corporate educational video distribution. Most of the 
present and planned packet networks, particularly Asyn- 
chronous Transfer Mode (ATM) networks, exhibit packet 
loss. Since compression reduces redundancy in the 
video signal, losing random parts of a compressed video 
bit-stream may cause drastic reduction in the quality of 
the received video. It is a continuing problem to improve 
the quality of video transmission ova such packet net- 
works. 

Summary of the Invention 

The present invention discloses apparatus fund 
method of transmitting a video bitstream from a transmit- 
ter ova a packet network to a receiver, the video bit- 
stream including a plurality of high priority segments and 
low priority segments. The high priority segments con- 
tain vital control information such as the codes that 
define the start of pictures and picture heada informa- 
tion. The low priority segments contain the rest of the 
bitstream. As an example, the high priority and low pri- 
ority segments may be determined in accordance with 
the well-known MPEG-2 draft standard (see Genaic 
Coding of Moving Pictures and Associated Audio Rec- 
ommendation H.262, ISO/I EC 13818-2, March 1994). 
According to the present invention, the transmitter and 
receiva locations use agreed-to high priority i n for ma t i on 
(e.g., predefined high priority segments, or format for 
generating same) and consequently only the low priority 
segments of the video bitstream need to be transmitted 
to the receiva. At a receiva location, the high priority 
segments are obtained (from storage or generated using 
the agreed-to format) and are interleaved with the 
received low priority segments in real time to recreate 
the video bitstream. 

Brief Description of the Drawing 

In the drawing. 

FIG. 1 shows a diagram of our inventive method for 

the compression, storage and transmission of low 

priority segments; 


FIG. 2 shows an illustrative client serva video dis- 
tribution useful in understanding the present inven- 
tion; 

FIG. 3 shows various formats for staring the previ- 
5 ously agreed-to high priority information at the 
serva location; 

FIG. 4 is a flowchart describing the serva and client 
interactions for handling video segment requests; 
FIG. 5 shows Table 7-28 of the MPEG-2 draft stand- 
io ard illustrating priority break points; and 

FIG. 6 shows an example of an agreed-to high pri- 
ority format 

Detailed Description 
is 

With reference to FIG. 1, we describe how the 
present invention is used tar transmitting low priority por- 
tions of video infor ma tion ova a packet network. Analog 
video signals associated with each video program (a 
so video segment) are received in the standard NTSC, PAL, 
etc., formats, digitized in digitizer 105, compressed in 
compressor 110 and the resulting compressed video bit- 
stream is stored in a disc 115. Illustratively; the well- 
known format of a video bitstream of a video segment is 
ss shown by 120. As shown, the illustrative video bitstream 
data for a video segment includes a plurality of high pri- 
ority (e.g., HP1) and low priority (e.g., LP1) segments. In 
an encoded bitstream, the partition boundaries (e.g., that 
between HP1 and LP1) cannot be determined without 
so some processing. This "data partitioning’ is described in 
the above-identified MPEG-2 standard as a technique 
that splits a video bitstream into two layers called parti- 
tions. A priority breakpoint indicates which syntax ele- 
ments are placed in partition 0, which is the base 
as partition (also called high priority partition). The remain- 
der of the bitstream is placed in partition 1 (which is also 
called low priority partition). The interpretation of 
"pri°rity_breakpoint" is given in Table 7-28 on page 113 
of the MPEG-2 standard, which is reproduced haein as 
40 FIG. 5. 

In the prior art, when a request for a video segment 
is received, the video bitstream 1 20 is obtained from stor- 
age 115 and processed by a priority transmitter unit (not 
shown in FIG. 1) so that the high priority partition (e.g„ 
45 segments HP1 -HPn) is sent ova a high priority channel 
of a transmission facility and the low priority partition 
(e.g., segments LP1 - LPn) is sent ova a low priority 
channel of the facility. In such an arrangement the high 
priority channel is typically one having a high probability 
so of delivery, while the low priority channel is one having a 
substantially Iowa probability of delivery. Undesirably, 
facilities which connect to a packet network may only 
have channels which operate at one error rate. In such 
an arrangement the prior art techniques may not be as 
ss useful. 

In accordance with the present invention, an appa- 
ratus for and method of sending a video bitstream, 
including a plurality of high priority se g ment s and asso- 
ciated lew priority segments, by transmitting only the low 
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priority segments from a transmitter over a facility to a 
receiver is disclosed. The transmitter and receiver loca- 
tions are arranged to use previously agreed-to high pri- 
ority information (e.g.. predefined high priority segments 
or format for generating same) and, consequently, only 
the low priority segments of the video bitstream need to 
be transmitted to the receiver. At the receiver, the high 
priority segments are obtained (from storage or gener- 
ated using the agreed-to format) and interleaved in real 
time with the received low priority segments to recreate 
an interleaved video bitstream. 

As shown in step 130 of FIG. 1. a server processor 
obtains the stored bitstream 1 15 and processes it using 
previously agreed-to high priority infor ma t io n (e.g., the 
format shown in FIG. 6) to generate the low priority seg- 
ments (also referred to herein as a low priority partition). 
In step 132, the agreed-to high priority information is not 
sent to the dient receiver; rather, in step 134, only the 
low priority segments are sent. 

With reference to FIG. 2, we describe an illustrative 
server/client network in which the present invention may 
be utilized. Such a network includes a server location 
200 which connects via packet network 210 to a plurality 
of dients 220-230. Server location 200 indudes a video- 
on-demand server/processor 201 which controls the 
operation of the hand disc 202 and network interface 203. 
The hand disc 202 is used to store the compressed video 
bitstreams. It should be noted that ail of the compressed 
video data stream need not be stored locally on disc 202, 
but may be remotely stored, as part of a remote fie 
server, and accessible via facility 204 (e.g., a local area 
network). 

The network interface 203 is utilized to transmit and 
receive using the protocols required by the packet net- 
work 210. The processor 201 performs the functions 
illustrated at the server portion of the flowchart of FIG. 4. 

The packet network 210 may be an Asynchronous 
Transfer Mode (ATM) network, an FDDI, an Ethernet or 
other similar packet networks. 

A dient location, e.g., 220, indudes a network inter- 
face 223 for converting communication protocols 
received from and transmitted over a facility connected 
to packet network 210. The video dient processor 221 
processes the previously stored agreed-to high priority 
information to generate the high priority segments which 
are interleaved in real time with the received low priority 
segments as illustrated in the flowchart of FIG. 4. A disc 
222 is used to store the generated high priority seg- 
ments. Processor 221 interleaves the high priority seg- 
ments with the low priority data received by network 
interface 223 and then sends the interleaved data to 
decoder 224. The output of decoder 224 is the requested 
video segment which is then displayed on the dient mon- 
itor 225. 

Similarly, cfient 230 includes processor 231, disc 
232, network interface 233, decoder 234 and monitor 
235. 

Shown by 310 is trie raw compressed video bit- 
stream data for video segments 1 - n, where the high 


priority and low priority portions are HP1 - HPn and LP1 
- LPn, respectively. In the raw compressed data stream, 
the separation between the high priority (e.g., HP1) and 
low priority (e.g., LP1) data portion can be defined based 
5 on priority breakpoints. These priority breakpoints can 
have a variety of values as shown in FIG. 5 which illus- 
trates Table 7-28 of the prariously referenced MPEG-2 
standard. With reference to 310 of FIG. 3, HP1 would 
indude sequence start GOP and picture header 1. The 
io picture header also indudes a time or frame identifier. 
The sequence start header indudes data defining the 
vertical and horizontal resolution of the video segment 
A video segment indudes multiple pictures (or frames). 
The picture (or frame) rate may be from 24 per second 
is for motion pictures to 30 per second for broadcast tele- 
vision. Each picture (or frame) may indude a plurality of 
slices. The GOP data is optional, and indudes the picture 
coding utilized for the segment The picture header 1 
indudes the header info r m a t i on for all slices of picture 1 
20 (or frame 1). The low priority data portion LP1 indudes 
the rest of the bitstream for picture 1. Additionally, in 
accordance with the present invention, the low priority 
data portion LP1 may indude time and frame identifiers 
corresponding to that of HP1 . Similarly, each of the high 
25 priority data portions HP2 through HPn indude at least 
the headers for the slices of the associated picture. The 
length of each video segment may be different and have 
a different number of high priority and low priority data 
portions within each segment Note that the size of the 
go low priority data portions LP1 - LPn may be different 
lengths due to picture content 

In accordance with the present invention, the previ- 
ously agreed-to high priority information, illustrated in 
Table 339, can be based on the breakpoint values 0 and 
as 1 shown in Table 7-28 (FIG. 5). 

The agreed-to high priority information nay take any 
of a variety of forms inducting, illustratively, a format 
derived from an international standard, a client/server 
agreed-to high priority format or a predefined high pri- 
*0 ority partition. When a video bitstream for each video 
segment is received, the network server 200 must then 
process the data stream using the previously agreed-to 
high priority infor ma tio n (e.g., using one of the formats 
shown in Table 339) to obtain the low priority segments 
45 associated with each video program. These low priority 
segments for each video segment are then stored as 
compressed video on disc 202. It should be noted that 
the agreed-to high priority information may use 0-1 ofthe 
priority breakpoints of Table 7-28 of FIG. 5. Illustratively. 
so in one embodiment the agreed-to high priority format 
indudes those data items of priority breakpoint 1 shown 
in Table 7-28 of FIG. 5. 

With reference to RG. 6, we illustrate an eocarrple of 
a previously agreed-to high priority standard format (341 
55 of Table 339) which may be used by server 200 and a 
dient As shown by 601, the format indudes an M and 
N number and MPEG-2 picture level information. As 
shown, the MPEG-2 picture level i n fo r mation indudes an 
I and, optionally, a P and/or B frame. As shown, the M 
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specifies the number of B frames between P or I frames 
+ 1 and N specifies the number of B or P frames between 
I frames + 1, where i is the intra-cod ed frame, B is the 
bidirectional coded frame, and P is the predictive coded 
frame. Shown in 602 are several examples of different M 5 
and N values tar high priority formats 1,2 and 3. 

Returning to FIG. 2, it should be noted that the 
processing of the bitstream data to obtain low priority 
segments may be done by server 200 or the taw priority 
segments may be received in processed form by server 
200 and downloaded to disc 202. 

With joint reference to FIGS. 2,3, and the flowchart 
of FIG. 4, we descrfoe the server and client interaction 
for handling video segment requests from a client 

In step 400, the client may either store the agreed- 
to high priority inform a tion (e.g., an algorithm or format 
for generating a high priority partition) or may optionally 
generate and store the high priority partition itself, bi step 
401, server 200 is in a wait state, waiting for a client 
request In step 402, a client 220 requests a particular 
video sequence or segment In step 403, network server 
200 receives the client’s video segment request 

In step 409, the server 200 sets the communication 
protocol for n on-guaranteed delivery of the low priority 
data of the requested video segment to the client 220. In 
accordance with an aspect of the present invention, the 
system can limit the number of times that the low priority 
data of the requested video segment can be sent to the 
client 220. This ability to multiply send the low priority 
partition 312 to the dient 220 enables tiie dient to have 
a capability to pause or rewind during the reception of a 
video segment 

In step 41 0, dient 220 sets a communication proto- 
col for the non-guaranteed reception of the taw priority 
partition. 

In step 411, server 200 reads from disc 202 the pre- 
viously generated taw priority data partition 312 or 
optionally may create the low priority data partition 312 
in real time based on the previously agreed-to high pri- 
ority information (of Table 339). hi step 412, server 200 
transmits the low priority data partition 312 in real time 
to dient 220. The packets containing the LP data may 
indude frame numbers, time stamps, sequence num- 
bers enabling resynchronization to the associated HP 
data at the dient decoder 224 in the case of LP packet 
loss. 

In step 413, dient processor 221 may optionally read 
the previously generated On step 400) high priority par- 
tition from memory or generate it in real time and, in step 
41 4, send it to video decoder 224. Instep415, dient 220 
receives the low priority data segment in real time from 
server 200 and sends it in step 41 6 to video decoder 224. 
The steps 41 4 and 41 6 are coordinated by processor 221 
so that the stored high priority partition is interleaved in 
real time with the received low priority partition sent to 
video decoder 224. The interleaved data would have the 
original raw compressed data stream format shown by 
310. Video decoder 224 then converts the interleaved 


data to the display format needed to enable monitor 225 
to display the requested video segment 

In step 417, dient 220 can request to replay the 
video segment If a control command request is made, 
in step 418, control returns to step 411. A control com- 
mand request may be the well-known VCR-type request 
induding stop, pause, fast forward, fast backward, replay, 
etc. The stop command instructs the server 200 to not 
send any additional low priority data. The pause com- 
mand performs the stop command function but addition- 
ally causes decoder 224 to hold the last frame. The tost 
forward command causes server 200 to send only 
MPEG-2 I frames of Ihe low priority partition. The fast 
backward command causes the server 200 to send the 
I frames in reverse order. The replay command causes 
server 200 to re-send only the low priority partition. 
These capabilities may illustratively be implemented 
consistent with the Trick modes' described in section D. 
12 at page 167 of the previously referenced MPEG-2 
draft standard. 

If the control command requests a new video seg- 
ment step 419, control returns to step 403. If no control 
command requests made, step 420, then processor 221 
clears the previously stored high priority data from mem- 
ory. In step 421 the procedure ends and control returns 
to the standby state, step 401. 

What has been described is merely illustrative of the 
application of the principles of the present invention. 
Other arrangements and methods can be implemented 
by those skilled in the art without departing from the spirit 
and scope of the present invention. 

Claims 

1. A method of transmitting a video bitstream from a 
transmitter (200) over a facility to a receiver (230), 
said method CHARACTERIZED BY the steps of 

to the transmitter, 

processing (130) the video bitstream using 
previously agreed-to high priority information to 
obtain low priority segments, and 

transmitting (134) a lav priority partition, 
induding the low priority segments, over the facility 
in real time using a packet delivery mechanism hav- 
ing a certain probability of success, and 
to the receiver, 

receiving (415) the low priority partition, 
obtaining (413) high priority segments using 
the previously agreed-to high priority information 
available to the receiver, and 

interleaving (416) the high priority segments 
in real time with associated received low priority seg- 
ments to recreate an interleaved video bitstream. 

2. The method of claim 1 CHARACTERIZED IN THAT 
the obtaining step obtains the high priority informa- 
tion (400, 413) which has been previously stored at 
the receiver as high priority segments. 
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3. The method of daiml CHARACTERIZED IN THAT 
the high priority infor mati on is a high priority format 
and wherein the obtaining step uses the format to 
generate the high priority segments. 

5 

4. The method of daim 3 CHARACTERIZED IN THAT 
the processing step includes the steps of 

receiving (1 20) a video bitstream having high 
priority segments interleaved with associated low 
priority segments, and io 

selecting (130, 132, 134) the low priority seg- 
ments for transmission as part of the low priority par- 
tition. 

5. The method of daiml CHARACTERIZED IN THAT 
the low priority partition includes tune stamps for at 
least some of the low priority segments and CHAR- 
ACTERIZED IN THAT, at the receiver, the interleav- 
ing step associates the high priority segments and 
the low priority segments using the time stamps 
received in the low priority partition. 

6. The method of daim 1 CHARACTERIZED IN THAT 
the low priority partition indudes frame identifiers for 
at least some of the low priority segments and 2 s 
wherein, at the receiver, the interleaving step asso- 
ciates the high priority segments and the low priority 
segments using the frame identifiers received in the 
low priority partition. 

30 

7. The method of daim 1 further CHARACTERIZED 
BY the steps of 

at the receiver, 

requesting a delivery of a bitstream of an 
identified video segment; and as 

at the transmitter, 

receiving the video segment request and in 
response thereto accessing the video bitstream for 
the identified video segment prior to the transmitting 
step. 40 

8. The method of daim 1 further CHARACTERIZED 
BY the steps of 

at the receiver, 

sending a contrd command request to the 
transmitter; and 

at the transmitter, 

decoding a received contrd command 
request and controlling the transmission of the low 
priority partition to the receiver in response thereta 

9. The method of daim 8 CHARACTERIZED IN THAT 

the control command request is selected from a 
group of commands induding at least stop, pause, 
fast forward, and replay commands. ss 

10. The method of daim 1 further CHARACTERIZED 
BY the steps of 

at the receiver, sending a replay command to 


the transmitter requesting a replay transmission of 
the low priority partition; 

at the transmitter, in response to the replay 
command, sending a replay transmission of the low 
priority partition to the receiver; and 

at the receiver, interleaving each generated 
high priority segment available at the receiver in real 
time with each received low priority segment of the 
low priority partition of the replay transmission 
received from the transmitter. 

11. A method of communicating a video bitstream, over 
a packet network, CHARACTERIZED BY the steps 
of 

processing (130) the video bitstream using 
previously agreed-to high priority information to 
obtain low priority segments; 

transmitting (134) the low priority partition 
over the network using a non -guaranteed delivery 
mechanism of the network; 

receiving (415) the low priority partition at the 
receiver location; and 

interleaving (416) segments of a previously 
agreed-to high priority partition available at the 
receiver in real time with associated segments of the 
received low priority partition to recreate the video 
bitstream. 

12. Apparatus for transmitting a video bitstream from a 
transmitter (200) over a facility to a receiver (230), 
the video bitstream induding a plurality of high pri- 
ority segments and associated low priority seg- 
ments, said apparatus CHARACTERIZED BY 

at the transmitter, 

means for processing (201, 202) the video 
bitstream using previously agreed-to high priority 
information to obtain low priority segments, and 

means for transmitting (201 , 203) a low prior- 
ity partition, induding the low priority segments, of 
the video bitstream over the facility in real ti me using 
a packet delivery mechanism having a certain prob- 
ability of success; and 
at the receiver, 

means for receiving (233, 231) the low priority 
partition, and 

means tor (231 , 234) high priority segments 
obtained using previously agreed-to high priority 
information available at the receiver in real time with 
associated low priority segments to recreate an 
interleaved video bitstream. 

13b Apparatus for transmitting a video bitstream over a 
fadlity to a receiver, said apparatus CHARACTER- 
IZED BY 

means for receiving (204) a video bitstream; 
means for processing (201, 202)the video bit- 
stream using previously agreed-to high priority infor- 
mation to obtain low priority segments; and 

means for transmitting (201 , 203) only a low 
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priority partition, including the low priority segments, 
of the video bitstream over the facility in real time 
using a packet delivery mechanism having a certain 
probability of success. 

5 

14. Receiver for reconstructing a video bitstream from 
information received over a facility from a transmit- 
ter. said receiver CHARACTERIZED BY 

means for receiving (233, 231) low priority 
partition information including low priority segments io 
from the transmitter; and 

means for interleaving (231, 234) high priority 
segments obtained using previously agreed-to high 
priority information available aft the receiver in real 
time with the received low priority segments to rec- is 
reate an interleaved video bitstream. 
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